Perceptual Aliasing in JCMB (or, Where on Earth is IPAB?)

In the real world we usually have to rely upon what we can observe about our environment in order to judge our current state. Unfortunately our observations are inherently limited, and this can sometimes cause us to become confused and disorientated, losing track of exactly what state we are in. This confusion is called perceptual aliasing, and it occurs when our observations are not descriptive enough to allow us to uniquely identify our state. When encountered by artificial agents this perceptual aliasing phenomenon can seriously damage the agents ability to learn solutions to a problem, and this project looks at a method of overcoming this problem whilst learning with reinforcement learning algorithms. The general method of overcoming perceptual aliasing considered is called active perception — this is giving the agent control over its own sensors, so that when it encounters an aliased state it can attempt to adjust them to gain new information that resolves the ambiguity. In this project a particular type of active perception known as perceptual actions is discussed, with results being presented that confirm it can help alleviate the effects of perceptual aliasing. It is noted that there are actually two minor variations of the perceptual action approach, and both are analysed to evaluate their comparative performance. Additionally a new grid-world problem for reinforcement learning agents, based upon Edinburgh University’s James Clark Maxwell Building, is introduced. This problem is designed to suffer from perceptual aliasing and is used as an additional test problem for the algorithms studied. Finally some brief, but promising, results are given for a new adaption of the above technique, in which the agent learns in multiple observation spaces simultaneously.